There has been known a method of generating an image having an arbitrary focal distance, depth of field, and viewpoint based on images obtained by photographing an object from many viewpoints.
For example, in NPL 1, there is disclosed such a configuration that a microlens array having microlenses that are arrayed at a rate of one for a plurality of pixels is arranged in front of an image pickup element so as to obtain not only two-dimensional distribution of light intensity but also information on an incident direction of light rays entering the image pickup element to thereby obtain three-dimensional information on an object space.
Such an image pickup apparatus as described above that is capable of obtaining the three-dimensional information on the object space is called “light field camera.”
In addition, the three-dimensional information on the object space is called “light field data,” and image processing of what is called “refocusing” can be performed by obtaining the light field data and performing thereon the image re-formation process after photographing. The refocusing includes, for example, changing of a focusing position of an image and a photographing viewpoint thereof and adjustment of the depth of field of the image.
When such light field data as described above that enables the refocusing is to be viewed, it is difficult in some cases to determine at which focal position an image to be displayed first is to be refocused. In order to solve this problem, in the technology of PTL 1, an object is extracted from the image to display a refocus image which is in focus at a position where the object is positioned.